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A gauge model of neural network is introduced, which resembles the Z(2) Higgs lattice gauge theory 
of high-energy physics. It contains a neuron variable S x = ±1 on each site £ of a 3D lattice and a 
synaptic-connection variable J x)1 = ±1 on each link (a;, x + jxj^fi = 1, 2, 3). The model is regarded as 
a generalization of the Hopfield model of associative memory to a model of learning by converting the 
synaptic weight between x and x + p, to a dynamical Z(2) gauge variable J XM . The local Z(2) gauge 
symmetry is inherited from the Hopfield model and assures us the locality of time evolutions of S x 
and J X fj, and a generalized Hebbian learning rule. At finite "temperatures", numerical simulations 
show that the model exhibits the Higgs, confinement, and Coulomb phases. We simulate dynamical 
processes of learning a pattern of S x and recalling it, and classify the parameter space according to 
the performance. At some parameter regions, stable column-layer structures in signal propagations 
are spontaneously generated. Mutual interactions between S x and J xfl induce partial memory loss 
as expected. 



I. INTRODUCTION 

Our brains exhibit various complicated functions like 
recognition, thinking, etc. In particular, it is quite in- 
teresting to understand the mechanism of storing and 
restoring particular concepts, i.e., how to learn and re- 
call them. It is challenging to construct a mathematical 
model to describe the essence of learning and recalling. 

In the framework of neural network, it is well known 
that the Hopfield model [1] of associative memory of- 
fers us a reasonable mechanism how to recall the im- 
ages and patterns that one has once learned. In the 
Hopfield model, the state of the i-th neuron, excited 
or unexcited, is described by the Z(2) variable Si(— 
±l)(z = 1,2, :;N), and the state of the synaptic con- 
nection between the i-th and j-th neurons is expressed 
by its strength (weight), Jy, to transmit signals, which is 
a preassigned real constant. The signal at the j-th site at 
time t propagates to the i-th site through the axon and 
the synaptic connection in the form JijSj(t) to affect the 
state Si in the next time step t + e as 



(not increase) the following "energy" , 



S t (t + e) =sgn 



JY 



(i.i) 



Physically, Sj is the signature of the electric potential 
at the j-th neuron (membrane potential) measured from 
a certain common level, and Jy is the conversion factor 
to propagate the potential from j to i. So the quantity 
y~]j JijSj in the bracket is just the total amount of po- 
tential accumulated at the i-th neuron. The case Jy > 
enhances the tendency that the i-th neuron shall take the 
same state as that at j, Si = Sj, while the case Jy < 
suppresses it, favoring the opposite state, Si = —Sj. 

The time evolution (1.1), when applied to all the vari- 
ables Si in an asynchronous manner, is known to decrease 



1 N 

E = — - SiJijS. 



(1.2) 



Let us prepare M patterns Si — £f (a — 1,2, ...,M) 
(which are mutually orthogonal, Yli£i£i = N5 a p) as 
the patterns to learn. One may choose Jij so that each 
of these patterns, Si — is a local minimum of this 
energy. Such an idea is realized by taking the following 
Hebbian learning rule [2]; 



1 M 

a=l 



(1.3) 



If we start from a pattern of S'i(O), then one of the mem- 
orized patterns that locates nearest to S'j(O) , say £?, is 
obtained gradually via the rule (1.1), Si(t) — £f in a 
certain time [3] . This certainly offers us a possible mech- 
anism of associative memory. 

How about processes of learning patterns? The Hop- 
field model itself is incapable for this purpose, since the 
memorized patterns are stored in the fixed parameters 
Jij as (1.3) from the beginning. We need some gener- 
alization so that these Jy change as time goes by, i.e., 
plasticity of Jij. A simple example of the time depen- 
dence of J^ is the following; 



J ij (t + e) = Jij(t) + 



1 

M 



(1.4) 



It reflects the Hebbian rule (1.3), describing the process 
that the pattern Si = £f is learned in the period (t, t + e). 

A well-known model of learning is a perceptron or its 
improvement, the error back-propagation model [4], In 
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a perceptron, one assigns a pattern Si — £j for neu- 
rons Si in the input layer and optimizes the weight W{ 
connecting Si and the single output neuron 5" so that 
S' = OiEi^iSi) produces 1(0) for # = &) ap- 

proximately. The model proves itself very useful for the 
problems of pattern recognition. However, in a percep- 
tron, (i) the flow of informations is one way from the 
input layer to the output neuron, and (ii) the time evo- 
lution of Si(t) is missing, although the process of opti- 
mization of Wi itself may be regarded as a time evolution 
of Jij. These two points seem quite different from what 
is going on in our brains. 

In a set of models of self-organization [5-8], signals 
from the neurons Sj in the input layer are forwarded 
with the weights Wij to the neurons S- in the output 
layer, and Wij(t) are assumed to evolve with the rule 
like Wij(t + e) — Wij(t) — —awij + ftS-Sj, i.e., with the 
damping effect and the Hebbian rule (and/or certain con- 
straints on Wij ) . It is shown that the neurons in the out- 
put layer are partitioned into subsets, and the neurons 
in each subset become active for a particular pattern in 
the input layer. This may explain the emergence of so- 
called column structures of active neurons observed in 
human brains like ocular dominance columns, orientation 
columns, and direction columns in the medial temporal 
area. However, these models also share the properties (i) 
and (ii) above. 

As a stochastic model of learning, Boltzmann machines 
[9] are well known. Here Si develops according to Boltz- 
mann distribution for the energy (fefhopfieldenergy) and 
the learning rule is like Wij(t + e) — Wij(t) = (3((SiSj)^ — 
(SiSj)E), where the first term in R.H.S. is an average 
over patterns to learn and describes the Hebbian rule 
whereas the second term is an average over Bolotzmann 
distribution and describes so-called unlearning processes. 

Following these pioneering works, various models of 
learning have appeared. Some models treat both Si(= 
±1) and Jij[& (—00,00)] as dynamical variables [10]. Al- 
though they contain certain interesting features, it seems 
necessary to consider some natural and explicit principles 
that characterize the dynamics of Si and Jij . 

In Ref. [11], a gauge model of neural network is pro- 
posed. Here Jij are regarded as gauge variables, i.e., 
path-dependent phase factors (exponentiated gauge field) 
and the energy of the model is obtained by adding a cou- 
ple of additional gauge-invariant terms to the Hopfield 
energy (1.2). Both Si and J^ are treated as dynamical 
variables and their time-dependences at finite "temper- 
ature" (T) are postulated to obey a stochastic process 
for Boltzmann distribution of the gauge-invariant energy. 
The model on a 3D lattice at finite T resembles the lat- 
tice gauge theory [12] in high-energy particle physics. 
The phase structure of lattice gauge theory was inten- 
sively examined by mean field theory (MFT) and Monte 
Carlo (MC) simulations [13]. Generally, three phases, 
Higgs phase, Coulomb phase, and confinement phase, are 
known to be possible [see the table (4.7) below]. In the 
gauge model of neural network, the former two corre- 



spond to the ferromagnetic phase ({Jij) 7^ 0, (Si) 7^ in 
MFT) and the paramagnetic phase ((Jij) 7^ 0, (Si) = 0) 
in the Hopfield model, respectively. The third confine- 
ment phase has (J^) = and (Si) = 0, describing the 
state that both learning and recalling are disable. For ex- 
ample, the Higgs phase corresponds to a smart student 
who took a lesson and made a good score in examination; 
the Coulomb phase to a student who took a lesson but 
with a bad score; the confinement phase to a student who 
cuts a class. 

There is a quite interesting relation between the 
present gauge model and the quantum memory of a toric 
code studied by Kitaev et al. [14]. To calculate the accu- 
racy threshold of a 2D toric quantum memory, one needs 
to study the phase diagram of the 3D pure Z(2) lattice 
gauge theory with random gauge coupling. The present 
model at c\ = C3 = (see Sect.IIIA) coincides this gauge 
model at vanishing randomness. The error-free condition 
in uploading data to and downloading from a quantum 
memory leads us that this pure Z(2) gauge model should 
be in the ordered (Coulomb) phase [15]. Thus the ability 
of leaning patterns in the gauged neural network corre- 
sponds to the error-free function of a quantum memory 
of a toric code. 

In this paper, we study various aspects of this Z(2) 
gauged neural network in detail. We investigate the full 
phase structure of the system at finite T. We also simu- 
late the dynamical processes of learning a pattern of Si 
and recalling it, and examine the parameter dependence 
of the rate of performance. 

Here we note that Thoulouse [16] noticed Z(2) gauge 
symmetry in the related models, the models of spin 
glasses [17], and adopted it to characterize effects of frus- 
tration. In fact, the Sherrington-Kirkpatrick model of 
spin glasses [17] has the same form of energy as that of the 
Hopfield model (1.2). Since then, there have appeared 
various studies on frustrations in spin glasses [18]. The 
crucial difference between the models of spin glasses and 
the present model of neural network is in that the "gauge 
variables" J^ are regarded as quenched random vari- 
ables in the spin-glass theory whereas they are treated 
in the present neural network as dynamical variables on 
an equal footing to the dynamical neuron variables Si. 

The structure of the paper is as follows; In Sect. II, we 
explain the origin and the relevance of Z(2) gauge sym- 
metry in neural networks in details. In Sect. Ill, we intro- 
duce the Z(2) gauge model on a 3D lattice. In Sect. IV, 
we study its phase structure at finite T by statistical me- 
chanics and find emergent column-layer structures. In 
Sect.V, we simulate dynamical processes of learning a 
pattern and recalling it. In Sect. VI, we present discus- 
sion and the problems in future. 
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II. Z(2) GAUGE SYMMETRY 

In this section, we explain the origin of the gauge sym- 
metry and its relevance to neural networks. 

Let us start with the Hopfield model (1.2). Here one 
may assign Si = 1 for the excited (fired) state of the i-th 
neuron and Si = — 1 for the unexcited state. Let us focus 
on the part of network consisting of the j-th neuron and 
the axon and synaptic connection starting from the j-th 
neuron and ending at the i-th neuron, and consider the 
following two states (a) and (b) of this part; 

state (a) : (Sj, Jij), 

state (b) : (S'j = -Sj,^ = -Jy). (2.1) 

Physically, they arc independent (different) each other. 
For example, (a) (Sj = I, Jij = J > 0) describes 
the excited neuron and the enhancing connection, while 
(b) (Sj = —l,Jij = — J < 0) describes the unexcited 
neuron and the suppressing (inhibitory) connection. In 
the Hopfield model, these two states (a) and (b) are de- 
generate; they have the same amount of energy (1.2). 
Actually, the energy Eij stored in the above part is 

Eij^^-SiJ'ijS^-S^-JijK-Sj) 

= -SiJijSj = Eij(a). (2.2) 

The state (b) is obtained from (a) by the following 
replacement of Sj and Jy; 

Sj — > Sj = VjSj, Jij — > Jlj = JijVj, Vj = — 1. (2.3) 

In lattice gauge theory [12], the replacement (2.3) is 
known local Z(2) gauge transformation in which 

Jij plays the role of a so-called gauge variable. More 
generally, one may consider the following general local 
(site-dependent) Z(2) gauge transformation throughout 
the network (for all i and j); 

Si^s'i^ViSi, j, j -./;, r../,,i;,. v, = ±i. (2.4) 

Here, whether one replaces the state (a) by (b) (Vi = —1) 
or not (V^ = 1) may depend site by site. It is easy to 
see that the Hopfield energy (1.2) is invariant under this 
transformation. 

We regard this "gauge symmetry" of the Hopfield 
model as an important property that the generalized 
model of learning should inherit. To generalize the Hop- 
field model to such a gauge model of learning, we just 
need (i) to convert Jij from constants to dynamical 
(gauge) variables, which arc to transform as (2.4), and 
(ii) to introduce a gauge-invariant energy. 

In Fig.l we illustrate the function of the gauge field 
A ll (x) in the conventional U(l) gauge field theory by us- 
ing the path-dependent phase factor, 

U(x,y)=exp(i [ dx'^(x')), (2.5) 



where P xy is a certain path connecting two spatial points 
y and x. It conveys informations about the relative ori- 
entation of two internal coordinates at x and y, and 
parallel-translates a vector ip(y) at y to x along a path 
from y to x, giving rise to a vector U(x,y)(p(y) at x. 

Since the strength Jij of the synaptic connection de- 
scribes the way how the electric signal at j is transformed 
to i, it is natural also from this gauge-theoretical point of 
view to identify J^ as U(x, y) in gauge theory. Then the 
resulting signal conveyed from j to i, JijSj, corresponds 
to U(x,y)tp(y). 

Let us explain the relation between the gauge sym- 
metry and the physical states in detail [20]. One may 
first think that the assignment of the gauge- variant vari- 
able Si = ±1 to the the physical (excited and unexcited) 
states is inappropriate, because a gauge transformation 
Si = —Si exchanges the excited state and the unexcited 
state. However, such an assignment brings no difficulties 
because Jij also changes. Let us start with the global 
frame of Si illustrated in Fig. 2a, where Si = 1(— 1) de- 
scribe the excited(unexcited) state of the i-th neuron for 
all i. Suppose that we make a local gauge transformation 
with Vi = 1 — 26ij, which implies that Sj = 1(— 1) de- 
scribes the unexcited (excited) state. See Fig. 2b. Such a 
capricious or perverse choice is "corrected" or "compen- 
sated" by the associated transformation J[- = —Jij, thus 
bringing no changes in physical content. Actually, the 
state at j is to be recognized at the neighboring observer 
at i by the combination JijSj. There holds the equality 
JijSj — J^Sj, which shows that the physical state at j 
does not change. 

From these considerations, we regard J^ as the path- 
dependent phase factors of the gauge group Z(2), hence 
they take Z(2) variables, 

Jij = ±1 £ Z(2) (2.6) 

in appropriate unit (which is to be supplied through the 
coefficients in the energy) . One may claim that Jij should 
take continuous real values — oo < Jij < oo instead of 
Z(2) variables. Such an assignment is certainly possible 
in constructing a gauge model, as long as Jij transform 
as (2.4). However, there should be certain constraints on 
Jij, which reflect, e.g., conservation laws of chemical ma- 
terials and restrict their possible values. On this point, 
study of the biological models of self-organization [5-8] 
is suggestive and can be taken as a support to our sim- 
plification (2.6). For example, von der Malsburg [6] puts 
the constraint J2j Jij = 1 f° r cacn while Tanaka et al. 
[7] argue that the values of J^ can be well approximated 
by the Potts spin variables; J,j = or l,J2j Jij = 1- 
While both authors focus on the enhancing connections 
(Jij > 0), Linsker [8] considers also the case of sup- 
pressing connections (Jij < 0) and puts the constraints 
— 1 < Jij < 1. Our treatment (2.6) may be viewed (i) 
as the special case of Linsker's treatment that most con- 
figurations are to reach the end points of his inequality, 
and/or (ii) as a generalization of Tanaka's result to the 
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Z(2) gauge theory in which both > and Jij < 
should appear as (2.4) indicates [19]. 

Concerning to the signature of . it was once thought 
that each neuron releases only one type of neurotransmit- 
ter (Dale's law), which implies that the signature of 
should be fixed for each j. However, there is now evidence 
that Dale's law does not hold, indicating neurons release 
more than one kind of neurotransmitter. Irrespective of 
experimental circumstances including this case, we have 
a sound reason to consider with indefinite signature. 
It is supplied by regarding the gauge model under con- 
sideration as an effective model of renormalization-group 
theory as explained in Sect.IIIA for the explicit lattice 
model. Each Si is associated there not with a single neu- 
ron but with a cluster of neurons, expressing the "aver- 
age" over the states of neurons contained in each cluster. 
Jij is also the average strength between two clusters, so 
its signature is to be indefinite. 

As stated before, the Z(2) gauge symmetry requires 
that the energy E({Si}, { Jij}) of the system is gauge- 
invariant; 



Si(=±l) 
J 4 ,(=±l) 



S'i^ 
V 



ViSu 



±i, 



(2.7) 



The principle of gauge symmetry puts severe restrictions 
on the form of E. Some possible gauge-invariant terms 
are depicted in Fig. 3. The first term SiJ^Sj in Fig. 3 is 
just the term of the Hopficld model. The second term 
SiJikJkjSj may be viewed to describe the combined ef- 
fect of the two successive processes SkJkjSj and SiJikSk- 
Actually, due to Sf, = 1, we have 



SiJikSk X SkJkjSj — SiJikJkjSj. 



(2.8) 



In the same way, the last two terms are interpreted as 

JijJji — SiJijSj X Sj JjiSi, 
Jij J jkJki — SiJijSj X SjJjkSk X SkJkiS%. (^-9) 



These two terms describe the combined processes taking 
place along closed circuits. Hebb [2] has once introduced 
the concept of so called reverberating circuits, in which 
short-term memories are considered to be stored. These 
terms (2.9) may describe functions of such circuits. 



III. MODEL 

A. Z(2) gauge theory on a 3D lattice 

To be explicit, let us formulate the gauge model on a 
3D cubic lattice. Then the system resembles to the lat- 
tice gauge theory introduced by Wilson [12], and known 
ample techniques are applicable to study it. We spec- 
ify each site by the site- index x and use \i = 1,2,3 as 
the direction index. We use fj, also as the unit vector in 



the fi-th direction. We set the lattice spacing a = 1 for 
simplicity. For each x we put a Z(2) spin variable, 



S x = ±1, 



(3-1) 



as Si with i <-> x, and for each link (xfi) = (x, x + /i), i.e., 
for nearest-neighbor (NN) pair of sites, we put another 
Z(2) variable, 



Jxfi 

as Jij with x <-» j, x + fi <-» i 
symmetric connections, i.e., Jj. 



(3.2) 

Below we consider the 
= Jji, so J xfl describes 



also the conductivity of signals propagating in the oppo- 
site direction from x + to x. J xfl is the path-dependent 
phase factor of gauge theory as discussed in the previous 
section. The local Z(2) gauge transformation is given by 





^s x = \ 


Jxfj. 


1 

* ^Xfl — 


v x 


= ±1. 



(3.3) 



As the energy E of the system, we propose 
E = —c\ Sx+^Jx^Sx 

X [I 

c 2 ^ ^ y ^ Jx(j,Jx-\-fi,iyJx-\-iy,(j,Jxiy 

X \1>V 

C3 y ^ y ^ [S x Jxi'Jx-\-i',i-iJx-\-fi.i'Sx+i_i 
~~\~SxJx — b'.l/Jx — b'.flJx — ls-\-fl. uS x+lx ). (3.4) 

Each term in (3.4) is depicted in Fig. 4. The term c\, 
which corresponds to the energy of the Hopfield model, 
describes the direct transfer of signals from x to x + fi. 
The term ci describes the self-energy of transfer of signal 
through the contour (x^x + [i^x + fi + v^x + 
v — * x) and the contour with the opposite direction as a 
reverberating circuit (2.9). The term C3 describes indirect 
transfers of signal from x to x + \i via the bypath, (x — > 
x + v — > x + v + (i — > x) as explained in(2.8). Each term 
of E is gauge invariant, so 



E({S X },{J\) = E({S X },{J^}). 



(3.5) 



Appearance of the reverberating C2 and indirect C3 
terms is also supported by the renormalization the- 
ory of critical phenomena. After a renormalization- 
group transformation to coarse-grain the system in the 
space and/or time scales by integrating out the short- 
wave-length/high-frequency modes of variables, every 
term in the energy of the resulting effective theory 
acquires corrections due to the integrated variables. 
These renormalization-group transformations also gen- 
erate terms that are not contained in the original energy. 
In our case, if one starts with only the ci-term, the Ci and 
c 3 terms, among other terms, are certainly generated as 
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these effective interaction terms. We note that these ef- 
fective terms are necessarily Z(2) gauge-invariant reflect- 
ing the gauge symmetry of the c\ term. This supports 
our postulate that the generalized model should respect 
the local Z(2) gauge symmetry that has been revealed in 
the Hopfield model. 

As stated in Sect. I, the present system is similar to the 
Z(2) lattice gauge theory [12]. There, S x is interpreted 
as a Higgs (matter) field and J XjU is the exponentiated 
gauge field, J xfl = exp(iA xli ). Its standard energy (ac- 
tion) is given by the first two (ci and c 2 ) terms; the C\ 
term represents the kinetic energy of Higgs particles in- 
teracting with the gauge field A xfl , while the c 2 term 
represents the energy of the gauge field. Actually, in the 
electromagnetic U(l) gauge theory, this c 2 term repre- 
sents the energy density E 2 + B 2 of the electromagnetic 
field [12]. The values of these coefficients Cj are related 
in principle with the energy of the original model that is 
defined at the microscopic level. This microscopic model 
is, however, not easy to specify, so we regard these Ci as 
effective parameters in a phenomenological point of view 
or in the sense of renormalization-group in the following 
discussion. This set of parameters may differ individu- 
ally, characterizing each neural network, i.e., each brain. 



B. Time evolution 

Let us consider the dynamics of S x (t) and J xll (t). We 
postulate that the energy E basically decreases as the 
time increases, which sounds natural since our brain is 
a dissipative system (as long as E is regarded as the 
physical energy). This rule of energy decrease may fail 
with some rates due to misfunctions of signal process- 
ing caused by noises, etc., and may be controlled by a 
parameter which we call "temperature" T G (0, oo); For 
higher (lower) T, failures occur more (less). This T should 
not be confused with the physical temperature of brain, 
although there may be some correlations among them. 

For a system of continuous variables, these postulates 
are implemented in Langevin equations of time evolu- 
tion [21]. For discrete Z(2) variables like S X ,J X ^ 7 we 
propose to use the Metropolis algorithm (MA) [22] as 
the rule of time evolutions. In fact, MA was adopted to 
study the dynamics of the Hopfield model in the name 
of Boltzmann machine [9] . MA is a standard algorithm 
to calculate the thermal averages (O) over Boltzmann 
distribution with the energy E, 

{0(S, J))^! °( S > J ) e*P{-0E(S, J)), (3 = i 



S,J 



z = J2^M-0E(s,j)). 

S,J 



T 
(3.6) 



By starting with a certain initial state {S x (0),J xll (0)}, 
MA determines {S x (£+1), J x „(£+1)} from {S X (£),J X „(£)} 



by a certain probabilistic rule, and generates a 
Markov(stochastic) process, 

{S x (0), J^(0)}, {S x (l), J xli {l)}, {S x (2), J x »(2)}, ■ ■ ■ 

(3.7) 

In Fig. 5, we present a flow chart of the MA for a system of 
Z(2) variables S. Then it is assured [22] that the following 
relation holds; 

M 

(0(S,J)) = lim ^ ({S^),J^)}). (3.8) 

i=l 

We regard the Markov process (3.7) just as the time evo- 
lutions of S x (t) and J xft (t) from t to t + e (e is a certain 
time interval) with the relation t = e£. Then the relation 
(3.8) means that our time evolution converges at large 
time to the equilibrium distribution given by (3.6). The 
time interval e is related with the MA parameters as, olj 
(written as a in Fig. 5), which control the rates of changes 
from S x {£) to S x (£ + 1) and from J xfl {£) to J xli {£ + 1), 
respectively; a — > 1(0) implies slow(fast) changes. 

As Fig. 5 shows, S x (t + e) and J x ^(t + e) are deter- 
mined by the energy difference like AE = E(—S x (t)) — 
E(S x {tj) oc dE/dS x and AE = E(-J xli (t)) - 
E{J xl j,{t)) oc dE/dJ xfl . These partial derivatives [23] are 
given by 



x n 



S x — nJ x - 



c 3 ^ ^ ^ ^ [J X vJ x +v,iiJ x +p,,i'S x -\-iA 

J X —v,vJ X —i',nJ X —i'-\-ii,i'& X -\-iJ, 
~t~ & x —ixJx—ix,vJ x -\-v—ix,ixJ X v 
4~ S X —ixJ X — v—n,vJ X — v—ia,nJ X — v,v): 



dE 

$ J XU 



= -ClS t 



X-\-(l<Jx 



- c 2 



{^Jx-\-ia,vJx-\-v,iaJxv ~t~ Jx — v+f-L.vJxfiJx — v,v) 



£-3 ^ ^ x — v J x — v .y J x-\-fi — v.y $x-\-\x— v 
~t~ S x +ijJ X vJ x +t_i,vS x +ii+i' ~~\~ S x J x J t ^i^J x J t u^f 1 S x J r i/ 

~t~ S X J X + ^i — V ^ V J X — v^S X — V -\~ S X -i r v-\-[ 1 J X -]- I ;^J X1 /S X -\-[i 
~t~ Stz — iz+nJ^z — i/^Jtz — i/.i/Sx + fx}. (3.9) 

From these expressions, we observe a couple of merits of 
the gauge-invariant energy; 

(i) The time evolution of S x is affected only through 
the terms containing J xfJl and J x -^^ that have contacts 
with S x by inspecting dE / dS x Likewise, the evolution of 
J xf i is driven by the terms that have contact points x and 
x + /i as one inspects dE/dJ xfl . The gauge symmetry 
assures us that the time evolutions take place through 
local interactions. As stated in Sect. II, this is a welcome 
property. 
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(ii) The first term in the right-hand-side of the second 
equation of (3.9) expresses just the Hebbian rule (1.3) to 
learn the present pattern S x for c\ > 0. The remaining 
terms in the right-hand-side describe the indirect effects 
by neighboring S x and J xfl , which generalize the Hebbian 
rule. 

The typical time scales of variations in S x (t) and J xll (t) 
may be different in general. Actually, it is widely ac- 
cepted that, in the actual processes occurring in human 
brain, the changes in Jjj is much slower than those of 
Si. One may take it into account by controlling the char- 
acteristic time scales of these two sets of variables by 
assigning different values for the parameters as as < olj. 
We come back to this point in Scct.V. 

IV. PHASE STRUCTURE 

In this section, we study the phase structure of the 
system with the energy (3.4) at finite T by using MA 
(3.7). In Sect.IVA, we formulate the MFT and present 
the resulting phase diagram together with the results of 
MC simulations. Some special cases of the model are 
also discussed. In Sect.IVB, we explain the details of MC 
simulations with MA. In Sect.IVC, we study the column- 
layer structure for C2 < 0. 



where N is the total number of lattice sites (we present 
the formulae for <i-dimensional lattice) as 

/„ = -|ki(2cosh/?W) - ^ ln(2 cosh/%) - Cl dm 2 M 

-c 2 d(d 2 1) M 4 - Ac 3 d{d 2 1) m 2 M 3 + dWM + hm, 
m = (S x )o = tanh/3/i, 

M= {V x „) =tanh/3W. (4.4) 

The stationary conditions df v /dW = df v /dh — read 

W = cito 2 + 2c 2 (d - 1)M 3 + 6c 3 (d - l)m 2 M 2 , 
h = 2dcimM + 4c 3 d(d - l)mM 3 , (4.5) 

which give rise to the equations for m and M; 

m = tanh [2l3dc x mM + 4(3c 3 d(d - l)mM 3 ] , 
M = tanh [f3 Cl m 2 + 2c 2 (3(d - 1)M 3 

+6pc 3 (d - l)m 2 M 2 ] . (4.6) 

By assuming suitable scaling behavior of parameters f3ci 
at large d, the result of MFT is known to become exact 
for d -> oo [25]. 

The MFT equations (4.4-4.6) generate the three phases 
characterized as follows; 



A. Mean field theory and the phase diagrams 

The MFT may be formulated as a variational method 
[24] for the Helmholtz free energy F; 

Z = U E II E exp(-/?£)^cxp(-/?F). (4.1) 

x S x =±l x,fi J X(J =±1 

Actually, for a trial energy Eg there holds the following 
relations; 

Zo = H E IT E exp(-/3S ) =cxp(-/3F ), 

x S x =±l x.fi J xfl =±l 

(O) = Z^H E II E OeM-PE ), 

F <F V = F + {E-E ) . (4.2) 

From this Jensen-Peierls inequality F < F v , we adjust 
the variational parameters contained in Eq optimally so 
that F v is minimized. 

For Eq of the present system, we assume the transla- 
tional invariance and consider the following sum of single- 
site and single-link energies; 

£o = -^EE J ^-^E^' (4.3) 

x ii x 

where W and h are real variational parameters. Then we 
calculate the variational free energy per site, /„ = F v /N, 



phase {Jxp.)o (S x )a ability Hopficld 

Higgs 7^ learn and recall ferromagnetic 

Coulomb 7^ learn paramagnetic 

Confinement N.A. N.A. 



(4.7) 

In the first column of (4.7), the name of each phase is 
given, which is used in particle physics. The second 
and third columns show the order parameters of MFT, 
(J XjU ) (= M), (S^o (= m) [26]. The fourth column 
shows the properties of each phase. The condition M/0 
is a necessary condition so that a phase has the ability to 
learn a pattern S x — by storing it to J x ^. There the 
fluctuations of J xfl should be small so that J xfi generate 
a nontrivial minimum of E at S x — (M = means 
the fluctuations are too large.) The condition m ^ is 
a necessary condition to recall the pattern S x — . The 
fluctuations of S x around S x = £f should be small when 
recalling is successful. We note that the fourth phase 
with M — and m ^ is missing. This sounds nat- 
ural since learning should be a necessary condition for 
recalling, hence partly supporting the gauge symmetry 
of neural network. The fifth column indicates the corre- 
sponding phases in the Hopficld model. 

In Fig. 6 we plot the phase diagrams obtained from (4.4- 
4.6) for various values of c 3 . (The case of c 3 = has been 
studied in Ref. [11].) The results of MC simulation in the 
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next subsection are also presented by filled circles. The 
phase boundary of MFT between the Higgs phase and the 
Coulomb phase is second order, while other two bound- 
aries, Higgs-confinement and confinement-Coulomb, are 
first order. In Fig. 7 we present typical behaviors of /„ 
for (a): second-order transition; (b) first-order transition. 
Across a second-order transition, M and m vary contin- 
uously, while across a first-order transition, M and/or 
m change discontinuously with finite jumps AM and/or 
Am. For a Higgs-confinement transition, AM ^ and 
Am 7^ 0, and for a confinement-Coulomb transition, 
AM ^ and Am = since m = in both phases. 

The locations of phase boundaries of MFT agree glob- 
ally with those of MC simulation in Scct.IVB. However, 
the results of MFT are not sufficient in the following two 
points; 

(i) The MC simulation shows that the confinement- 
Coulomb transitions are second-order contrary to the 
MFT. This point may be explained as d = 3 is not large 
enough for MFT. The MC simulation for d = 4 [27] for 
c 3 = reports first-order confinement-Coulomb transi- 
tions as the MFT does. 

(ii) The MC simulation shows that the Higgs- 
confinement boundary terminates at a certain end point. 
Along this phase boundary, the jumps AM, Am decrease 
as c 2 decreases and disappear at the end point at c 2 > 0. 
This corresponds to the complementarity studied in Ref. 
[28] for c 3 = 0, which states that these two phases are 
analytically connected through a detour. 

There are the following special cases that are examined 
better by other methods; 

CASE I: Ising model (c 2 — > 00) 

Here the c 2 terms forces J x>1 to the so-called pure-gauge 
configuration, and the energy reduces to that of the Ising 
model; 

-(ci+4c3) £2 + const, 

S' x = V X S X = ±1. (4.8) 

Thus, there is a second-order Ising transition at /3(ci + 
4c 3 ) ~ 0.22 for c 2 = 00. 

CASE II: Pure gauge model (ci = c 3 = 0) 
Here the energy is 

E = -c 2 2 2 J xnJx+n,vJx+v,yi,Jxv + const. (4.9) 

This system is known [27] to exhibit a second-order tran- 
sition at /3c 2 ~ 0.76. Actually, after the duality trans- 
formation, the system (4.9) is converted to the 3D Ising 
spin model. 



CASE III: Single-link model (c 2 = c 3 = 0) 

Here the sum over J xfl is possible because the energy is 
decoupled to each link, and the partition function be- 
comes an ^-independent constant; 

2 exp((3dS x J Xfl S x+tl ) = 2cosh(f3c!S x S x+f ,) 

= 2cosh(/?ci), (4.10) 

due to S% = 1. Thus the free energy has no singularities 
and no phase transitions along the ci-line. This explains 
why the Higgs-confinement transition line should trun- 
cate at the end point (before reaching the ci-line) as the 
complementarity [28] claims. 

CASE IV: Self-duality curve (c 3 = 0) 

For c 3 = 0, the standard Z(2) duality transformation can 
be applied [29], which maps the model at P(/3ci,/3c 2 ) 
onto the same model at P'(/3ci, (3c' 2 ) with 

13c', = -^nth(/3c 2 ), /3c 2 = -^nth(/? Cl ). (4.11) 

If we assume the phase transition occurs at the point 
satisfying P' = P, we obtain the curve of phase transition 
from (4.11) as 

/tea = -^nth(/fci). (4.12) 

In the next subsection, we shall see that a part of the 
curve (4.12) is actually the phase boundary. 

B. MC simulation 

We performed MC simulations for a 3D lattice of the 
size N = L 3 up to L = 16 with the periodic boundary 
condition. The case of ci = c 3 = has been examined 
by Bahnot and Creutz. [27]. We employed MA, which is 
illustrated in Fig. 5, with choosing the prefactors as = 
olj = 0.9. Typical numbers of sweeps [M of (3.7)] are 
10 5 for thermalization and 5 x 10 4 for measurements. 

Among others, we measured the internal energy U and 
the specific heat C, 

U = (E), 

C=^=p 2 ((E-(E)f). (4.13) 

We judge the order of transition as follows; If U has a 
discontinuity at the transition point, it is of first order. A 
typical behavior of U and C for a first-order transition is 
given in Fig. 8a and Fig. 8b. If U is continuous and C has 
a peak and discontinuity, then it is of second order. A 
typical behavior of U and C for a second-order transition 
is given in Fig. 9a and Fig. 9b. 
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Let us comment on the gauge fixing. Our MC simula- 
tions have been done without fixing the gauge. We have 
also made MC simulations in the unitary gauge, 

S x = 1. (4.14) 

Although the partition function in (4.1) and the corre- 
sponding averages are independent whether one fixes the 
gauge or not, variations of the variables in Markov pro- 
cesses in the unitary gauge are too small and the con- 
vergence of the expectation values are too slow to obtain 
meaningful results with good accuracies. This slow con- 
vergence shall persist even for other gauge fixings. 

C. Column-layer structures in C2 < 

Usual studies of lattice gauge theory have been re- 
stricted to the case of c 2 > 0. However, as a model of 
neural network, the case of c 2 < is also interesting since 
this condition implies that the C2-term in the energy ex- 
presses negative feed backs, that is, signals starting from 
x will propagate around the plaquette and return to x 
with a negative coefficient. 

For c 2 < 0, the c 2 -tcrm prefers "flux states" of J xfl , 
i.e., J X vJx+v,nJx+n,vJxn = -1- Then it will generally 
compete with the ci-term which prefers fluxless states, 
JxvJx+v,nJx+n,vJxn = 1 [31]. Thus the system is frus- 
trated. The MFT with the translationally-invariant vari- 
ational energy of (4.3) is inadequate for this situation, 
and the MC simulation is necessary. 

In Fig. 10, we present the phase diagram of MC simu- 
lation for the extended region with c 2 < and/or ci < 0. 
In the region c 2 < 0, there arc phase boundaries that 
look like mirror images of c 2 > of Fig. 6 except that 
the Higgs-confinement boundaries extend to c\ — > ±oo 
instead of terminating at the ending critical points. The 
phase F in Fig. 10 corresponds to the Higgs phase B, the 
phase E to Coulomb phase A, and the region D is con- 
nected to the confinement phase C. For C3 = the phase 
diagram is symmetric w.r.t the c\ = 0-line, reflecting the 
symmetry under (01,03) <-» (— Ci, — C3) [32], i.e., 

Z(ci,c 2 ,c 3 ) = Z(-a,c 2 , -c 3 ). (4-15) 

Bahnot and Creutz [30] performed MC simulations for 
c 2 < at c 3 = with the related interest of spinglass 
(though taking J xfi as an annealed variable). Our result 
at C3 — is consistent with their result. At C3 7^ 0, the 
symmetry w.r.t. the c\ = 0-line is violated as shown in 
Fig. 10b for /3c 3 = 0.05. 

In the "Higgs" phase F, we observe a "column-layer" 
structure in the spatial configurations of the following 
link objects; 

jx^ = (S x +nJ x ^S x ). (4.16) 

j Xfl is the thermal average of a gauge invariant extension 
of J xfx as appeared in the ci-term of the energy. It mea- 
sures the efficiency of signal propagations between x and 



x + /U. In Fig.ll, we present the basic unit of a typical 
periodic configurations of j xfi that forms a column-layer 
lattice in the phase F. 

This column-layer structure on the lattice is stable, be- 
cause j Xfl are not snap shots but thermal (i.e., time) aver- 
ages in long MC runs. It is to be spontaneously generated 
as a result of dynamics, i.e., a result of self-organization 
in the present model. In short, its origin is the frustra- 
tion (competition) between the ci and c 2 terms in the 
energy. 

There are several possible configurations of j Xfl which 
differ from that of Fig.ll in the direction of ID align- 
ments (i.e., other than in the 1-direction) and the ori- 
entation of the planes of layers (other than in the 1-2 
plane). These options are generated by starting with 
different initial configurations (and/or different random 
numbers). In this sense, their generations are sponta- 
neous, but they are stable once they are formed as we 
explained. 

Since a large j Xfl means that signals (potential) prop- 
agate between S x and S x +^ frequently and coherently, 
each column in the column-layer structure is a ID path 
along which signals propagate dominantly. It is interest- 
ing to study possible relevance of such a structure to the 
actual self-organized structures observed in the human 
brains like ocular dominance columns. [5-8] (See the fu- 
ture problems in Sect. VI.) We also note that the present 
structure of weakly-coupled planes may be viewed as a 
kind of multilayer neural network. 

V. LEARNING AND ASSOCIATIVE MEMORY 

In the previous section, we studied the phase structure 
of the model, which reflects the static properties of the 
model. In this section, we simulate the processes of learn- 
ing a pattern of S x and recalling it. The results reflect 
the dynamical properties of the model. 

We set up the simulations in the following two steps in 
time; 

(i) Learning during < t < t\\ 

We first prepare a pattern S x = £ x to memorize and start 
with it, S x (0) — £ x . During this learning time, we freeze 
S x (t) as 

S x (t) = S x {0) for < t < h, (5.1) 

by hand to let the system learn it. This may correspond 
to apply very strong stimuli to the brain like forcing it 
to watch the pattern with concentration. On the other 
hand, we allow J xfl (t) vary according to MA to adjust 
themselves to a configuration suitable for S x (0). In terms 
of the prechoice parameters as, aj of MA (where 1 — a is 
a parameter to control the rate of time variation) , we set 
as = 1 and aj < 1 for < t < t\. Below we fix aj = 0.9 
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for < t < t\. The time t is defined throughout the sim- 
ulations (0 < t) as t = (1 — a, ;(0 < t < ti))x number of 
iterations (sweeps), i.e., t = O.lx number of iterations. 

(ii) Recalling during t\ < t: 

At t = ti we change S x discontinuously from S x (0) to 
a pattern 5 x (ii) that is obtained by adding a certain 
amount of random noise upon S x (0). For definiteness, 
we set S x (ti) = — S x (0) for the 10% of sites x that are 
randomly chosen throughout the entire lattice (except 
for Fig. 17 discussed later where we consider the cases of 
more amounts of noises) . This forced change of S x (t) at 
t = t\ simulates relaxing the brain from the concentration 
upon S x (0) and letting it forget S x (0) partly. Then we let 
S x (t) vary according to MA with as < 1- Below we fix 
as = 0.9 for t\ < t. On the other hand, J xll (t) basically 
vary according to MA with a,j < 1 smoothly starting 
from J XjU (ti). However, with the reason explained below, 
we shall be also interested in the special limit a,j — > 1 in 
which the time variations of J x „(t) after t\ freeze; 

olj -► 1 : J xft (t) = J xfl (h) for ti < t. (5.2) 

To judge the performance of each process of learning a 
pattern and recalling it in a quantitative manner, we use 
the following overlaps; 

Os(*) = ^£S x (0)S x (i), 

X 

x ^ 

which are gauge-invariant under (3.3). If the recalling is 
successful, we expect S x (t) ~ S x (0) at sufficiently large t, 
so O s ~ 1. (Note that O s (ti) = 0.8 for the 10% change 
of S x at t = ti.) On the other hand, Oj(t\) measures 
the rate how much the synaptic connections change by 
learning during < t < t\. In Fig. 12, we illustrate Os 
and Oj versus t in two typical processes for the case 
(5.2). Fig. 12a is a process that succeeds in recalling, and 
Fig. 12b is one that fails in recalling. 

Let us first study the dependence of the overlaps upon 
the choice of a tJ for t x < t. In Fig. 13 we plot Os{t) and 
Oj(t) for three cases, aj = 1.0, 0.9, 0.99 for t 1 < t. (Note 
that we fixed aj — 0.9 for < t < t\.) As the initial 
condition, we choose 5 X (0) and J xp (0) randomly. We se- 
lect three points in the parameter space of Ci. Fig. 13a is 
the result for ci = 1,C2 = C3 = which belongs to the 
confinement phase. For aj = 1 (J XjU (i) = J xtl {ti)), Os 
approaches to the constants near 0.95; the system (al- 
most) succeeds to recall S x (0). For aj = 0.99, Os once 
increases and then decreases. It describe a process of re- 
calling 5 X (0) partly and then lose the memory gradually, 
that is, the phenomenon of a partial memory loss (deficit). 
For a j = 0.9, O5 decreases monotonically. These latter 
two curves exhibit a typical phenomenon of a dynamical 
system with coupled variables. A fixed point obtained by 



fixing some variables may become unstable when all are 
coupled. As expected, Os decreases more as J xfi change 
more rapidly (i.e., aj decreases more). Fig. 13b is for 
the point c\ = 3.0, c 2 = 1.0, c 3 = 0, and Fig. 13c is for the 
point ci = 3.0, c 2 = —1.0, c 3 = 0, both of which are in the 
Higgs phase. In contrast to Fig. 13a, Os in these points 
have smaller dependence on a j , so the effects of memory 
loss are smaller than in the confinement phase. Although 
the time- variations of J xfl (t) for t\ < t (aj ^= 1) reduce 
the performance of recalling more (in the deconfinement 
phase) or less (in the Higgs phase), we stress that these 
phenomena of partial memory loss are not a flaw of the 
present model, but a welcome feature that the realistic 
model of human brain should possess. 

Since the case of aj = 1 for t\ < t gives the sta- 
ble and highest values of Os at large t, we present the 
results for aj(t\ < t) = 1 in Fig. 14-18 [33] below as typ- 
ical results for aj ~ 1. This choice aj{t\ < t) = 1 for 
presentations sounds also reasonable because the actual 
variations of synaptic connections, J xtl (t), in the human 
brain are much slower than those of S x (t) as explained 
in Sect.IIIB, i.e., aj ~ 1. (In this viewpoint, our choice 
aj = 0.9 for < t < t\ corresponds to accelerate the 
learning processes significantly. If we take larger aj ~ 1 
for < t < t\, then we need to take t\ larger in order to 
obtain the same values of Os) 

Next we study the dependence on the initial condi- 
tions. In Fig. 14 we present the contours of Os at large 
t for four choices (a-d) of the initial states S x (0), J xtJ ,(0). 
The results of first three choices (a-c) look similar each 
other, while the fourth case (d) 5^(0) = 1, J xfJ ,(0) = 1 is 
different; the region of higher Os for C2 > extends to 
the phase boundary (c\ ~ 0.4) of the Higgs phase and the 
Coulomb phase. This is natural since the configuration 
S x = J xfJ , = 1 is the minimum of the free energy in the 
Higgs phase at c 2 > 0. Thus the system has learned this 
pattern 5a; (0) = 1 from the beginning. Below we present 
the result for the random initial condition, the choice (a), 
i.e., S x (0) and ^^(0) are chosen randomly from ±1. 

To study the dependence on the learning time t\, we 
present in Fig. 15 two typical processes with different t\. 
In Fig. 15a, t\ — 1 and Os approaches to Os — 0.6, so 
it fails to recall S x (0). On the other hand, in Fig. 15b, 
<i = 5 and Os approaches to Os — 0.92, so we judge it 
succeeds (but not completely) to recall 5^(0). A reason 
is clearly drawn from the behavior of Oj. In Fig. 15b, 
Oj almost converges to a fixed value at t\, that is J xpi 
converge to the suitable configuration, while, in Fig. 15a, 
Oj is in a way to converge. Thus, a necessary condition 
to learn successfully is to keep t\ sufficiently long (longer 
than the relaxation time of J xp ) so that J xfl can converge 
to the configuration that makes S x = S x (0) a minimum 
of the energy. For definiteness we present the results for 
t\ = 5 below. 

Let us next study the relative importance of each term 
ci, C2, C3 of the energy in learning and recalling. In Fig. 16, 
Os after sufficiently large time is plotted for three cases 
where only one Cj of c\ , c 2 , c 3 is nonvanishing and the 
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other two are zero. In the case of c\ alone, as c\ in- 
creases, Os approaches unity. This is expected since the 
Ci-term describes the direct transfers of signals. In the 
case of C3 alone, as C3 increases, Os increases but sat- 
urates around Os — 0.65. This indicates that the indi- 
rect signal transfers by the C3-term is not sufficient by 
itself to recall the original pattern, as anticipated. In the 
case of c 2 alone, Os ~ 0. This is natural because the 
C2-term, which contains only J xfl and no S x , expresses 
signal-transfers starting from a neuron and ending at the 
same neuron through a loop along a plaquette, but de- 
scribes no signal-transfers to the NN neurons. 

Let us see the roles of C2,C3-terms in details. Since 
we have observed that the Ci-term plays the central role 
in learning and recalling, we simulate the processes with 
Ci + c 2 and c\ + c 3 . In Fig. 17 we plot Os after large time 
for several S x {ti)'s prepared by adding certain amounts 
of noises to S x (0) from 10% up to 40%. It shows that 
adding certain amounts of c 2 or c 3 upon C\ improves the 
rate of recalling, i.e., to achieve larger Os- It is inter- 
esting to note that there is a preferred region for c 2 , 
(3c 2 £ (0.2,0.6), for which O s is over 95%. This reminds 
us the phenomenon that applying a certain but not too 
much amount of magnetic field improves our brain activ- 
ities. 

Let us study the relation between the results of present 
section and the phase structure of the previous section. 
In Fig. 18, we superpose contour plots of Os on the phase 
diagram Fig. 6b. We observe that being in the Higgs 
phase is not sufficient to achieve good rates of success- 
ful learning and recalling. This is clear especially in 
the region with large c 2 , which is consistent with the 
case of c 2 alone in Fig. 16. Also, in the region near 
(3c\ = 0.5, /3c 2 = 0.4 which is a vague border between 
the confinement and the Higgs phases, learning is possi- 
ble. So this region may have something to do with our 
experience that a tiny amount of stimulation helps us to 
recall certain patterns; a coexisting phenomenon of re- 
calling and nonrecalling. Apart from these regions, there 
are certainly the correlations among the results of this 
section and the phase diagram of Sect. III. 

From these results, one may list up the conditions to 
succeed in learning a pattern of S x and recalling it as 
follows; 

- The learning time t\ should be larger than the relax- 
ation time of J xfl . 

- The temperature T should be low. 

- The self-interaction term /3c 2 should be within a certain 
range (0.0 < (3c 2 < 0.7 for /?ci = 1.0). 

- The indirect C3-term should be of the same signature 
as the direct ci-term to accelerate signal transfers caused 
by the Ci-term. 



VI. CONCLUSIONS AND FUTURE PROBLEMS 

By converting the strength of the synaptic con- 
nection to a Z(2) gauge variable (exponentiated gauge 
connection) and imposing the gauge symmetry on the en- 
ergy E({Si}, { Jij}), we have proposed an explicit model 
of neural network of learning. The Z(2) gauge symmetry 
is inherited from the Hopficld model. Study of the phase 
structure and the simulations of learning and recalling 
revealed several interesting features of the model. 

- In the confinement phase, both learning and recalling 
are disabled, which may corresponds to certain symptoms 
like Alzheimer's disease [See (4.7)]. The complementar- 
ity characterized by the end point of Higgs-confinement 
phase-transition curve may offer us some methods to re- 
trieve the network from the confinement phase to the 
Higgs phase in a continuous and practical manner. 

- The column-layer structure of (Sx+^Jx^Sx) discussed 
in Sect.IVC implies that there are particular ID paths 
(columns) in each layer along which the signals (electric 
potential) propagate dominantly. This structure seems to 
exhibit the potentiality of the present model to evolve the 
self-organized column structures of the active neurons, 
which are observed in the human brains [5-8]. 

- Due to the mutual interactions between S x and J Xfl , 
the phenomena of memory loss are observed as in Fig. 13. 

- The time evolution of J xll generalizes the Hebbian rule 
of learning. Suitable amounts of the newly added c 2 , c 3 - 
terms improve the performance of learning and recalling 
(Fig. 17). 

We think that these features make the present model 
interesting as a neural network of learning and associative 
memory and worth to make further investigations. 

There are certainly various ways to improve the present 
model. Among others, we list up the following; 

(1) Actual synaptic connections are not symmetric, 
i.e., 7^ Jji. However, in the Hopfield model, due to 
the very form of its energy E of (1.2), the antisymmetric 
part of J^, J^ — Jji, does not contribute to E. This flaw 
can be removed in the present framework by introducing 
two independent Z(2) variables, J xfl and J x +^,-^, on a 
link (x, x + n). Then the c 2 and C3 terms can reflect the 
antisymmetric part J xfl — J x +^-^. In fact, an asymmet- 
ric model in this direction is proposed in Ref. [11] (Model 
III) and its phase diagram is studied in MFT. 

(2) In the present model, synaptic connections arc 
restricted only to the NN neurons. In human brain, 
each neuron receives signals from 1000 ~ 10000 neurons. 
These long-range connections are certainly responsible 
to store many patterns and should be incorporated in a 
more realistic model. We plan to increase the number 
of connections in the future study. This means to in- 
troduce more variables Jij beyond NN pairs. Related to 
this point, one may increase the number of patterns to 
memorize as in the Hopfield model. 

(3) Natural ways to incorporate external stimuli like 
visual and acoustic ones to the present model may be (i) 
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to change the boundary condition from the present pe- 
riodic boundary condition to a fixed boundary condition 
with an appropriate values of S x , J xfl on the surface of 
the 3D lattice, and/or (ii) fixing S X ,J X ^ in certain part 
of the system to certain constants. By studying the re- 
sponse to each stimulation, one may address the question 
whether column structures of active neurons are gener- 
ated in the present model. 
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Note added 

The gauge model of neural network has been extended 
to a quantum neural network having U(l) gauge sym- 
metry in Y. Fujita and T.Matsui, Proceedings of 9th In- 
ternational Conference on Neural Information Process- 
ing, ed. by L.Wang et al. (2002)1360-1367 (arXiv:cond- 
mat/0207023). 
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xyPy 




FIG. 1. Function of gauge variable U xy (path-dependent 
phase factor). U xy parallel-translates a vector <p y at the 
point y to another point x giving rise to U xy ip y . To com- 
pare two vectors ip x and <p y , one needs to refer to the com- 
mon local frame, say the frame at x. So one should par- 
allel-translate (p y to x and take the gauge invariant scalar 
product (ip x ,U xy <p y ) = <plU xy <p y instead of (<p x , ip y ) = <pt<p y . 
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FIG. 2. Illustration of the local frames of Si. Each solid 
line with an arrow indicates Si(= ±1) at the site i (filled 
circle). Each dashed line indicates the gauge variable Jij 
which measures the relative orientation of local frames at 
nearest-neighbor points, (a) Localframes where all the frames 
are in the same orientation, (b) Local frames after the local 
gauge transformation at j, S'j = —Sj. The information of this 
change is stored(absorbed) in the neighboring gauge variables 
Jij so that the signal received at i is unchanged J[jS'j = JijSj. 




FIG. 3. Examples of gauge- invariant terms. The black 
circles denote Si and the curves with arrows denote Jij. The 
figures indicate SiJijSj, SiJikJkjSj, JijJji, JijJjkJki from 
the left, respectively. The first SJS term is just the term of 
the Hopfield model. The second SJJS term may be viewed 
to describe the two successive processes SkJkjSj and SiJikSk- 
The last two terms consist of J's only, and describe closed 
circuits, which may be taken as "reverberating circuits". 
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S x Jx/j S x+/I 



x + LI 



(a) 



x — v 



X 



x -f- LI 



X X + LI 

(b) 

X X + ll 



FIG. 4. Graphical representation of each term in E of (3.4). 
The black circles represent S x and the line segments represent 
Jx^. Fig. a describes the direct interaction between S x and 
Sx+n through Ja; M . Fig.b is taken as the smallest "reverber- 
ating circuit". Fig.c describes the combined effects of three 
successive processes given by Fig. a. 




AE E E(-S(t)) - E(S(t)) 



S(t+e) = -S(t) 



S(t+e) = S(t) 



S(t+e) 

FIG. 5. Flow chart of Metropolis algorithm for a system 
with Z(2) variables S = ±1 to determine the value S(t + e) 
starting from S(t). ran#l, ran#2 are random numbers dis- 
tributed uniformly in the interval (0, 1). This update process 
is to be done for each variable for every time step. In the 
present model, each update at every time step sweeps out all 
S x (t) , Jxii (t) throughout the entire lattice. 
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Phase Diagram (/3c 3 = 0.1) 



0.2 
0.1 


-0.1 
-0.2 
-0.3 
-0.4 





MFT 




MC1 16? site) — — 

Higgs 


Confinement ] 






\ Coulomb 



0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 

/)C 2 



(d) 



13 



FIG. 6. Phase diagram by MFT and MC simulation, (a) 
(3c 3 = -0.05, (b) I3c 3 = 0.0, (c) f3c 3 = 0.05, (d) f3c 3 = 0.1. 
MC simulations show that the Higgs-Coulomb transition and 
the confinement- Coulomb transition are second order, while 
the Higgs-confinement transition is first order. The MFT 
predicts the first-order confinement- Coulomb transition incor- 
rectly. Also the Higgs-confinement boundaries terminate at 
certain critical points instead of extending to C2 = 0. The 
dashed curve in (b) is the phase-transition curve (4.12) pre- 
dicted by the duality transformation. It almost agrees with 
our MC result in the period 0.55 < /3c2 < 0.75, in which the 
assumption P' = P for phase transitions is verified. 
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(a) 0(E) /N vs 0a. 



0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 

PC, 

(b) C/N vs 0a. 

FIG. 8. Internal energy and specific heat per site versus 
f3c\ for a typical first-order transition (c 3 = 0.0, /3c2 = 0.7). 
In the measurement, /3ci is increased first (filled circles) and 
then decreased (crosses). 
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FIG. 7. Typical behavior of free energy f v of (4.4) near 
phase transitions. (a) /3fv(m,M) versus m = (S x ) for 
(3c\ — 1,C3 = near the second-order transition at 
/3c2 — 0.166. We set M — (Jxy) at the value on the tran- 
sition point, M = 0.999. (b) (3Af v = /3(f v (m, M) - f v (0, 0)) 
versus M for /3c2 = 0.1, c 3 — near the first-order transition 
at which (3d = 0.678, m = m c = 0.989, M = M c = 0.648. 
The curves are drawn along the line m = (m c /M c )M in the 
(m, M) plane. The three curves are for /3ci = 0.66, 0.678, 0.70 
from above. 
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(a) (3(E) /N vs pa. 
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(b) C/N vs (3c 2 . 

FIG. 9. Internal energy and specific heat per site versus (3c2 
for a typical second-order transition (f3c\ = 0.1, c 3 = 0.0). In 
the measurement, /3c2 is increased first (filled circles) and then 
decreased (crosses). 
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FIG. 10. Phase diagram including the extended regions 
C\,C2 < for (a) C3 = and (b) /3c3 = 0.05. The phases 
D,E,F in C2 < are "mirror images" of C,B,A in C2 > 0, re- 
spectively. Crosses in (b) indicate the points at which j xll = 0. 
(For the case (a), j xil = along the line /3ci = 0.) 




FIG. 11. Basic unit of the column-layer structure in the 
"Higgs" phase F in C2 < 0. The entire configuration of 
j x/1 = (Sx+^Jx^Sx) is just the repetition of this 2x2x2 struc- 
ture in every direction. The thickness of each segment rep- 
resents the magnitude of j xfl - Each plane with a certain ori- 
entation (the 1-2 plane here)has a one-dimensional columnic 
alignment of links with large (thick) j xfl (say, in the 1-direc- 
tion in the upper plane). In the next (middle) plane, the di- 
rection of the columns with large j xfJ , is rotated by 90 degrees 
(in the 2-direction), and so on. Each pair of successive planes 
are weakly coupled by small (thin) jx^ (in the 3-direction). 
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(b) 

FIG. 12. Simulation of learning and recalling. For 
< t < ti, S x (t) is fixed to S x (0), and the system learns the 
pattern S x (0) by changing J Xfl (t). At t = ti, we disturb S x 
to S x {ti) discontinuously, where S x (ti) is obtained by adding 
a certain amount of random noise to S x (0)- For t\ < t, the 
system tries to recall Sx(0) by changing S x (t). (We illustrate 
the case that J xll (t) for ii < t is fixed to J xl i(ti).) (a) It 
succeeds to recall S x (0) with S x (t) ^ S x (0) and Os — 1- (b) 
It fails to recall S x (0) with S x (t) / S x (0) and O s ^ 1. 
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(a) (3ci = 1.0, c 2 = c 3 = 0. 
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(c) 0q = 3.0, /3c 2 = 1.0, c 3 = 0. 

FIG. 13. Os{t) and Oj(t) in simultaneous time variations 
of S x and J XI1 for ti < t (ti = 5). We choose aj = 0.9 for 
t < ti, and a s = 0.9, aj = 1,0.99,0.9 for tj < t. The point 
of a in the case (a) locates near the Higgs-confinement phase 
boundary in MFT, while the cases (b) and (c) locate in the 
deep Higgs phase. Oj's for three aj in (a) and (b) are almost 
degenerate, while Oj's in (b) are for aj — 1,0.99,0.9 from 
the above, respectively. 
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(b) 5,(0) = 1; J xll (0) random. 
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(c) S x (0) = 1; J xfi (0) 
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(d) 5,(0) = 1; J^(0) = 1. 

FIG. 14. Contour plots of O s at large t in the /3c2 — /3ci 
plane at C3 = for four different initial configurations (a)-(d). 
Three curves in each figure are contours of O s = 0.9,0.8,0.7 
from the above, respectively. The cases (a),(b),(c) look simi- 
larly each other, while the case (d) differs from them. 
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FIG. 15. Overlaps O s ,Oj for /3ci = 1.0, c 2 = 0.0, c 3 = 0.0 
with two different learning times ti; (a) ti = 1 and (b) ti = 5. 
The system fails to recall ^(O) in (a) with a small ii, while 
it succeeds in (b) with a suffciently large ti . 
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FIG. 16. Os at large t with the three cases; ci only 
(c2 = C3 = 0), C2 only (ci = C3 = 0), and C3 only 
(ci = c 2 = 0). 
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FIG. 17. Os at sufficiently large time with several S x (ti)'s 
produced by adding various amounts of noises upon ^(O). 
(a) (3ci = 1.0, c 3 = 0.0. (b) /3a = 1.0, c 2 = 0.0. 
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FIG. 18. Contour plot of Os in the /3c2 — /3ci plane at 
ca = 0. Three curves are contours of O s = 0.9, 0.8, 0.7 from 
the above. The phase boundaries of MC simulations in Fig.10 
are superposed. 
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